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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 
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About  200  man-portable  robots  deployed  in-theatre 

•  Foster  Miller  Talons  •  Remotec  Mini-Andros 

•  iRobot  Packbots  •  EOD  Performance  Inc. 

•  Mesa  Matildas  Vanguards 


Missions 

•  Explosive  ordnance  disposal  (EOD) 

•  Scouting  unsecured  bunkers,  buildings,  caves 


ALL  ARE  STRICTLY  TELE-OPERATED 


ALL  HAVE  PROPRIETARY  VENDOR-SPECIFIC  OCUs 
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■  A  significant  performance  trade-off  exists 

•  How  well  do  the  robots  assist  users  in  performing  their  jobs? 

•  How  much  does  the  OCU  interfere  with  the  users’  abilities  to 
perform/survive? 

■  Solution 

•  Make  robots  more  functional  with  greater  autonomy 

•  Minimize  or  eliminate  the  OCU 

•  Make  robots  more  desirable  for  alternative  tasks 
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Introduction 

-  Near  Term  Goal:  Provide  a  Common  OCU 

•  Simplifies  training  and  logistics  support 

•  Promotes  interoperability 

■  Long  Term  Goal:  Completely  Eliminate  the  OCU 

•  Ongoing  development  efforts  to  equip  warfighter  with 
Voice-Over-IP  technologies: 

-  Land  Warrior,  Raytheon  Systems  Company 

-  Advanced  Robotic  Controller  (ARC),  Exponent 

-  Others 

•  Supports  bi-directional  audio,  video,  maps 

•  This  same  HW  provides  all  that  is  required  to  control 
the  robot 
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WARFIGHTER’S  ASSOCIATE  CONCEPT 

■  Sophisticated  robotic  system 

■  Accompany  warfigher  during  mission 
execution 

■  Synergistic  teaming  of  human  and  machine 

■  Uses  existing  communications  equipment 

■  No  dedicated  OCU  required 
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SYNERGISTIC  HUMAN-MACHINE  TEAMING 


Human  Characteristics 


Machine  Characteristics 
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ENVISIONED  ROLES  OF  WARFIGHTERS  ASSOCIATE 


■  Complement  and  safeguard  the  human  partner 

■  Handle  any  high-risk  tasks 

■  Perform  mundane  and  tedious  tasks 

■  Serve  as  subject  matter  expert  on  demand 

•  Field  interpret/translate  foreign  ianguage 

•  Provide  repair/maintenance  instructions 

•  Oversee  field  medical  procedures 

•  With  internet  access,  provide  limitiess  knowledgebase 
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Introduction 

CONCEPT  DEVELOPMENT  PLATFORM 

■  Leveraging  existing  laboratory  prototype 

■  Sophisticated  navigation,  collision 
avoidance,  mapping,  and  surveillance 
schemes 

■  90-amp-hour  battery 

■  Extensive  self  diagnostics 

■  Size  compatible  with  SICK  laser 


V 

SPA  WAR 
Systems  Center 
Sen  Oiegc 


Towards  a  Warfighter’s  Associate  \/ 

SPA  WAft 

_  _  ,  m  mm  Systems  Center 

Technical  Challenges 

-  Mobility 

■  Power 

•  Many  different  options 

•  Improved  power  sources 

(wheels,  tracks,  legs) 

will  be  required 

•  Other  development  efforts 

•  Other  efforts  addressing 

addressing  this  issue 

this  issue 

-  Navigation 

■  Command  &  Control 

•  Localization  &  collision 

•  Need  to  reduce  operator 

avoidance 

control  burden 

•  Fairly  robust  solutions 

•  Major  thrust  of  our  effort 

currently  available 
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Command  &  Control 

In  the  beginning,  there  was  a  one-to-one  correspondence 
between  the  robot  and  some  dedicated  controller 

OPERATOR 

1 

ocu 

I 

ROBOT 

This  is  evolving  to  multi-robot,  multi-operator  complexities 
OPERATOR  OPERATOR  OPERATOR  OPERATOR 


ROBOT  ROBOT  ROBOT  ROBOT 
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■  Eliminated  prior 
one-to-one 
robot/CPU 
association 

■  Allowed  multiple 
robot  control 
including  robots  of 
different  types 

OPERATOR 


MDARS  Multiple-Resource  Host  Architecture 


ocu 


ROBOT  ROBOT  ROBOT 
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Common  OCU 

■  One  operator  can  control 
multiple  robots  as  with  MRHA 

■  Multiple  operators  can  now 
control  same  robot 

■  Will  support  handoff  from 
team  to  team 

■  Plug-n-play  modular  I/O 

SSC-SD’s  Common  OCU,  which  runs  the 

Multi-Robot  Operator  Control  Unit  (MOCU)  OPERATOR  OPERATOR  OPERATOR 


ocu  ocu  ocu 
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ROBOT 
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Common  OCU  Hardware 
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Handheld  unit 

m 


Top 


Bottom 


I  Snap-on 
Controller 


Features: 

•Common  Display  Unit 
•Snap-on  controllers 
(left  and  right) 

•6.4"  Sunlight  readable 
display 

•Real-time  video  display 
•Robot  status  display 
w/  gauges 
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Sample  OCU 
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LONG  TERM  GOAL:  ELIMINATE  OCU  ALTOGETHER 
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OPERATOR 

1 

OCU 

Natural-Langijage  Interface 
ROBOT 


OPERATOR 

1 

^  OpU  ^ 

NatujpJ-^'’ngi|age  Ner^ce 
ROBOT  ROBOT  ROBOT 


OPERATOR  OPERATOR  OPERATOR 

1  1  1 

OCU  OCU  OCU 

Natu^LcigL|age  Interfape- 

ROBOT 
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Bidirectional  Voice  I/O  Requirement 

■  Generate  speech  (speech  synthesis) 

•  Very  mature  technology 

•  Unlimited  vocabulary 

■  Understand  speech 

•  Recognize  spoken  words 

•  Parse  resultant  text 

•  More  difficult  to  do 
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■  Recognition  Algorithm:  Incoming  Speech - ^Text  String 

■  Commercial  Voice-Recognition 

•  Exploits  high  signal-to-noise  ratio  w.r.t.  incoming  audio  stream 

•  Speak  directly  into  a  boom  mike/telephone  mouthpiece 

•  Pre-taught  voice  signatures 

■  Battlefield  Environment 

•  Much  noisier  unstructured  environment 

•  Increase  in  voice  pitch,  speed,  volume  when  under  stress 
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Bone-conduction  Headsets 


•  Microphone  as  well  as 
earphones 


•  Acoustically  coupled  with 
the  bone  structure  of  the 
skull 


Jawbone 

•  Two  audio  microphones 

•  Bone-conduction  sensor 

•  DSP  to  subtract 
background  noise 


'  > 


Jawbone 
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Text  Parsing 

■  SIMPLISTIC  CASE:  Translating  structured  text  string  into  a 
command  using  layered-menu  conditional-branching  approach 

Name  of  Robot 
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i 

Action  1 


1 

Action  2 


r 


■] 


r 


■] 


Parameter  1  Parameter  2  Parameter  1  Parameter  2 
MORE  DIFFICULT  CASE;  Parsing  lengthy  unstructured  text 
WARFIGHTER’S  ASSOCIATE:  Needs  are  in  the  middle 

•  Semi-structured  text 

•  Fairly  short  phrases 
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■  Developed  prototype 
unstructured  text  parser 

•  Accepts  dictated  or  typed 
text  input 

•  Email  interface  allows 
independent  development 

-  Decouples  speech- 
recognition  errors 

-  Provides  useful  robot- 
human  interface 

ROBARTIII 


Email  sent  to  and  received  from  ROBART  III  requesting  an  image. 
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Requires  some  common  frame  of  reference 
to  which  both  human  &  robot  can  relate 

■  Robo-Centric  -  Relative  to  robot  itself 

■  Vision-Centric  -  Relative  to  robot’s  camera  view 

■  Model-Centric  -  Relative  to  absolute  world  model 


COMBINATION  OF  THESE  APPROACHES 
PROVIDES  THE  MOST  FLEXIBILITY 
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Relative  to  robot  itself 

•  Send  low-level  motion  commands  (turn  left,  right) 

•  Initiate  sensor-assisted  motion  (follow  the  wall  to 
the  left) 

Control  camera  gaze  (pan  left,  pan  right) 

Too  restrictive  and  operator  intensive  in  general 
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■  Relative  to  robot’s  camera  view 

■  Operator  looking  at  video  feedback 

•  Provides  direction  based  on  common  video  image 

•  “Enter  doorway  in  front  of  you” 

•  Vision  system  looks  for  and  highlights  doorway 

■  Can  also  illuminate  attributes  of  interest  with  laser  pointer 
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Vision-Centric  Reference 
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■  Very  appropriate  for  high- 
level  weapon  control 

•  Database  of  targets  and 
their  vulnerabilities 

•  Search-and-destroy: 

-  Identifies  object 

-  Zooms  in  on  vulnerability 

-  Non-lethal  weapon  cued 


Search  Algorithm  on  ROBART  III 
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■  Relative  to  an  absolute  world  model 


■  Outdoors -GPS 

•  MOCU 

•  Operator  draws 
desired  route 

•  Alternatively 
selects  waypoints 
or  goal  destination 


MOCU  Display 
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Model-Centric  Reference 
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Indoors  -  GPS  not  available 

•  Must  localize  using  range 
data  from  surroundings 

•  MDARS-I  had  a  priori  map 

-  Executes  virtual  paths  to 
predefined  nodes 

-  Not  practical  in  tactical 
applications 


/WD/AHS-/ patrolling  Camp  Elliot  Warehouse  in  1995. 
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SRI  International 

•  No  a  priori  knowledge 

•  SLAM 
INEEL 

•  Collision  Avoidance 


•  Grow  to  “3D” 
representation 

•  Awkward  to  send 
coordinates  (i.e.,  go  to 
1200,  3250) 
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Autonomously  exploring  Battery  Woodward 
(an  underground  WWII  bunker)  at  SSC-SD 
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Keality-Virtuality  Continuum  as  proposed  by  Milgram 


Environment  Reality  Virtuality  Environment 

Real  Environment  -  Robot’s  video  feedback 

Virtual  Environment  -  Represented  by  SLAM  model 

Augmented  Reality  -  Link  additional  info  to  video  image  as  pop-up  overlay 

Augmented  Virtuality  -  Link  additional  info  from  other  sensors  (or  from 
operator)  to  SLAM  model 


Pitch 
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Model-Centric  Reference 

Augmented  Virtuality  -  link  additional  sensor  info  to  SLAM  model 


XX 


homcstar 


Itdiuoivn 


Spm 


Virtual  model  fused  with  video  image  input  from 
video  sensor  onboard  the  exploring  robot. 


Virtual  model  fused  with  room  identification 
tags  from  operator  input. 
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Providing  high-level  (natural-language)  direction  implies  the 
robot  must  be  able  to  execute  high-level  autonomous 
behaviors. 

■  What  do  we  have  working  now? 

■  What  are  we  going  to  do  next? 

■  What’s  further  down  the  road? 
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Autonomous  Behaviors 

Working  Now: 

■  Basic/high-level  mobility 

■  Weapon  control 

commands 

■  Convoy  capability  video 

■  Explore  &  map  structure 

■  Progressive  sleep  mode 

■  Building  sweep 

■  Automated  recharging  or 

■  Surveillance 

refueling 

■  Motion  detection 

■  Self  diagnostics 

■  Target  following  video 

■  Send/receive  email  with 
attachments 
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Autonomous  Behaviors 
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Focus  in  FY-05: 

■  Simplistic  sign  interpretation 

■  GPS  waypoint  navigation 

■  Visual  landmark  homing 

■  Motion  detection-on-the-move 


SSC-SD 


Perceptek 
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Investigating  further  down  the  road: 
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•  Complex  sign  interpretation 

•  Face  recognition 

•  License-plate  capture 

•  Foreign-language  interpretation 
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Conclusion 

■  Need  more  autonomous  robotic  functionalities 

•  Technology  Transfer  program  improves  functionality 
and  autonomy  of  candidate  platforms. 

■  Need  to  lessen  control  burden  on  operator  by 
minimizing/eliminating  the  OCU 

•  Common  OCU  (standardized,  modular,  plug-n-play) 

•  Natural-language  interface 

•  Common  frame  of  reference  incorporating  robot-,  vision-, 
and  model-centric 

•  Expansion  of  SLAM  model  for  augmented-virtuality 
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